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I, Roland M. Hochmuth, a co-inventor of the above-referenced patent 
application, enclose hereto as Exhibit A a true copy of an invention disclosure form, 
with dates and portions redacted as noted, which was received by the Legal- 
Intellectual Property department of Hewlett-Packard Company on a date prior to 
September 21, 2001, in the ordinary course of business as part of Hewlett-Packard 
Company's invention disclosure program, and which indicates a conception of the 
invention which is the subject of the above-referenced patent application on a date(s) 
prior to September 21 , 2001 . 

The invention that is the subject matter of the above-referenced application 
was conceived and reduced to practice while working in the United States for 
Hewlett-Packard Company. The invention disclosure form attached hereto as Exhibit 
A includes an explanation of the subject matter of the claims of the present 
application. Specifically, at least pages 3-8 of the invention disclosure form include 
an explanation of the subject matter of the claims of the present application. 
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I, Roland M. Hochmuth, a co-inventor of the above-referenced patent 
application, also enclose hereto as Exhibit B a true copy of a correspondence, with 
portions redacted as noted, received from counsel pertaining to the preparation of 
the above-referenced patent application which was received on September 27, 2001. 
In response to receiving a draft of the above-referenced patent application from 
counsel, I reviewed the draft application and provided to counsel, for incorporation 
into a final draft of the above-referenced patent application, my comments, additions, 
changes, etc., which indicates, alone and/or in combination with the invention 
disclosure form, diligence in the completion of the invention which is the subject of 
the present Application from a time prior to September 21, 2001, continuously up to 
the date of filing of the above-referenced patent application. 

I hereby declare that all statements made herein of my own knowledge are 
true and that all statements made on information and belief are believed to be true; 
and further that these statements were made with the knowledge that willful false 
statements and the like so made are punishable by fine or imprisonment, or both, 
under Section 1001 of Title 18 of the United States Code, and that such willful false 
statements may jeopardize the validity of the application or any patent issued 
thereon. 
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authorized, prepared, and sub mitted to the Government 



Descriptive Title of Invention: 
Network Attached Graphics Adapter 
Name of Project 
Scalable Computing Systems 
Product Name or Number. 
IVA (Internet Visualization Architecture) 



Was a description of the invention published, o r are you planning to publish? If so, the date(s) and publication(s): 
Redacted 

Was a product including the invention announced, offered for sale. sold, or is such activity proposed? If so. the date{s) and location(s): 
Redacted 

Was the invention disclosed to anyone outside of HP. or will such disclosure occur? If so. the date(s) and name(s): 
Redacted 

Was the invention described in a lab book or other record? If so. please identify (lab book #. etc.) 



Was the invention built or tested? If so, the date: 
Redacted 



Was this invention made under a government contract? It so, the agency and contract number. 

Redacted ^ . 

Description of Invention: P/ease preserve all reco rds of the invention and attach addtoi? pages for the following. Each amonm page should 

be sianed and dated by the inventor® and witnesses). , oe . 
A. Description of the consWion and operation of the invention 0nclude appropriate schematic, block. & bmmg diagrams; drawngs, samples. 

graphs; flowcharts; computer listings; test results; etc.) 
B Advantages of the invention over what has been done before. 
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Description A ^ , 

T *rtTe^ 

™ e Xor^ mote network attached devices such as a compute node with a network tnterface card. Tte 
"thin clients" or "appliances". 

Current display refresh on the graphics adapter works as foUows: 

While (TRUE) 

If start of vertical retrace 

For all pixels in frame-buffer 

Read pixel from frame-buffer 
Convert pixel to output format 
Transmit out analog/digital port 

The functionality listed above would be replaced with the following: 

( If^w ) update is requested a time threshold has been reached 

If there has been a change in some portion of the frame-bufler 
If compression of the frame-buffer enabled 

tl^lo^Uer to Internet Protocol (IP) or other network protocol 
Transmit the compressed frame-buffer out the network interface. 



Else 



Convert the frame-buffer to IP or other network protocol 

Transmit the uncompressed frame-buffer out the network interface port. 



^^^^^^^^^^^ 

tiling on the image and keeping track of the tiles that have been modified. 

The aleorithm listed above could also be modified slightly to handle capabilities such as overlay planes and 



functionality. 



While (TRUE) . , 

If new update is requested a time threshold has been reached 

If there has been a change in some portion of the primary display surface or overlay planes 
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If compression is enabled 

Coalesce and compress the buffer resulting from the primary and overlay planes 
And cursor location. 

Convert the compressed buffer to Internet Protocol (IP) or other network protocol 
Transmit the compressed frame-buffer out the network interface. 



Else 



Convert the coalesced buffer to IP or other network protocol 
Transmit the uncompressed buffer out the network interface port. 



The operation of coalescing involves reading the pixel from the higher priority buffer or determining the buffer 
that has precedence and performing color LUT conversions and also merging m the cursor. 

One of the disadvantages of current display refresh technology is that the data that is sent out the frame-buffer is 
not tagged to allow delivery of the frame-buffer contents to multiple destinations. Consequently, graphics 
adapted are usually not used in "server" like environments where multiple users share a sin gle machme^ As a 
resXtechnologies like Citrix Metaframe are used that send the graphics protocol over the network which must 
be rendered by the client. 

There have been graphics cards created with multiple video output ports. With these types of video cards At 
might be possible to have one user using one region of the frame-buffer and another user using another region 
of the frame-buffer. It may also be possible to load a computer with more than one graphics adapters and have a 
one-to-one mapping between users and graphics adapters. However, the number of physical ports on the 
graphics adapter and number of graphics cards in the system limits these approaches. 

Usine this invention it becomes possible for multiple users/clients to share or time-slice a graphics card and 
EX£ orlque session and/or desktop. Given a graphics adapter with it' s own video memory asociate a 
unique region of the graphics adapter memory with each unique network device. For example if a graphics 
adapter has 32 Mbytes of memory the first 4 Mbytes might be assigned to network device 1 the second 4 
Mbytes Zf be assigned to network device 2, and so on. The algorithm listed below would then be used to 
update each network-attached device: 

WWle ( lSupdate is requested by a specific network device or a time threshold has been reached 
If the frame-buffer for the specific network device has been modified 

Configure the frame-buffer compression engine to compress the frame-buffer 

for the network device. 
Compress the frame-buffer for the network device. 

Convert to IP ., 
Configure the network interface unit to send the compressed frame-buffer to the 

network device. 

Transmit the compressed frame-buffer to the network device out the network 
Interface port. 

Another possible use of this invention is in the area of collaboration where multiple users share a common 
desktop or window within a frame-buffer. In this scenario the same frame-buffer or same region of the frame- 
buffer would be compressed and transmitted to multiple network-attached devices. 
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Advantages over Prior Art 

. Off-loads the compression, read back, and transmittal of the frame-buffer from the host system or CPU. 
Currently to transfer an image rendered in a graphics adapter to a remote network attached device the 
frame-buffer would be required to be read back from the graphics card into system memory, 
impressed, and sent over the host system's network interface. Alternatively, a PCI board or appliance 
could be used as identified in the invention disclosure called "Internet Graphics Appliance . However, 
this invention has additional benefits beyond the IGA approach, which are listed below. 
. Transmittal of the contents of the frame-buffer only occurs when the contents of the frame-buffer 
change Current graphics adapters transmit the entire contents of the frame-buffer to the monitor at a 
coSous refresh rate independent of whether or not any pixels within the frame-buffer have changed 
in value This is necessary because monitors and other display devices do not have their own local 
display refresh memory. As a result, the overhead of display refresh imposes a constant drain oil t the 
graphics adapters memory sub-system, which significantly impacts graphics rendering performance 
since the same memory is being accessed by the rendering engine and display refresh umt. Tbs 
invention decouples the actual display device refresh from the graphics adapter which could free up 
significant memory bandwidth when actual frame-buffer image updates are occurring >ss often than the 
Refresh rate of the display device. It is noted that if a network fabric such as Ethernet were used for 
display refresh, the capability presented here would be dependent on the remote network attached 
2 or display device to be able to persist the transmitted frame-buffer in its own local memory and 
to refresh from a local memory source. Currently this technology does not exist and this is the subject of 
another invention disclosure that is in the process of being submitted. 
• Transmittal Of the contents of the frame-buffer can be limited to regions of interest. Current video output 
portsTn graphics card transmit the entire contents of the frame-buffer at a ^—0^ rate 
such as 75 Hz independent of the portions of the frame-buffer that have been modified. Since the 
SaphTcs adapter renders the pixels into the frame-buffer it should be able to easily keep track of the 
Sents of the changes or tiles or regions that have changed within the frame-buffer very easily. The 
graphics adapter needs to only transmit the regions that have changed. 
. Transmittal of frame-buffer contents or some other region of the graphics adapters locd memory could 
LTaggeXddressed and sent to more than one network attached device. This wou d allow multiple users 
of aTaphics card to be supported simultaneously with their own session or in a collaborative 
enZlent. Current video output ports on graphics cards transmit the contents of the frame-buffer out 
th" port to whatever is plugged into it. The contents of the frame-buffer are not tagged and 
transmitted to a destination. This makes it very difficult to support multiple users or multiple display 
devices simultaneously using the same graphics adapter. 
. Lower system cost and higher efficiency can be obtained in comparison to the mvention disclosure 
"Internet Graphics Appliance" (IGA) or PCI board solution. The IGA is a separate appliance that grabs 
the Loming frame-buffer at refresh rates and performs many operations on the miming P*^** 
to identify if the current frame has changed and the regions or tdes that have changed. These changes 
c^uld be more easily and efficiently tracked by a graphics adapter and could be used to identify when 
data in the frame-buffer needs to be transmitted and the regions that are required to be transmitted. 
Addifionally a separate device with it's own frame-buffer memory and electrical components would not 
i^SvAJ^d mrther reduce the overall system cost. It is also noted that tins capability does 
not currently exist within a graphics adapter. The invention disclosure titled 
Appliance" presents a methodology that would allow the frame-buffer or region of the 
be set over the network to a remote display based on technologies and approaches currently avadable. 
The IGA invention could be viewed as a technology that could be used while transitioning from current 
technologies to the approach identified in this invention. 
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• Graphics adapter frame-buffer can be viewed remotely anytime, anyplace, and anywhere with no 
distance limitations. Current video out technologies has distance limitations and cannot be viewed from 
any network-attached device. 

• Graphics Adapter with Frame-Buffer Compression Engine and Network Interface. 

Possible Embodiments 

This section describes possible embodiments of the invention: 

• The frame-buffer compression engine and networking functionality may be implemented in a separate 
chip or chips on the graphics adapter as shown in Figure 1 Variation 1. The frame-buffer compression 
engine and networking functionality may be implemented in a custom ASIC chip, DSP, FPGA or other 
alternative. It may be possible to easily design a graphics board with this type of functionality on it 
without modifying the currently available graphics chips. 

. The frame-buffer compression engine and networking functionality may be implemented as a block or 
blocks within the graphics chip as shown in Figure 2 Variation 2. In this design scenario current 
graphics chips would have to be modified significantly to add this type of functionality. 

• The video and DVI port may be eliminated from the graphics adapter completely as shown in Figure 3 
Variation 3. This would reduce the cost of the graphics adapter. 

• The PCI or AGP interface may be eliminated from the graphics adapter in which case the graphics 
commands themselves would be received from the network interface as shown in Figure 4 Variation 4. 
However, the network connection would have to be significantly faster than what is currently available. 

• The PCI or AGP interface could be substituted with another type of network interface such as 
Infinniband or Fiber Channel or another type of network technology that may become available. This 
embodiment is not shown in a diagram, but is easily created by substituting in the diagrams everywhere 
where is states PCI or AGP. 

. Graphics Adapter with Frame-Buffer Compression Engine Only: In this embodiment the graphics 
adapter only has the frame-buffer compression engine and does not have a network interface. In this 
embodiment the graphics card compresses the frame-buffer. The compressed frame-buffer would be sent 
back to the host system and transmitted over the network by the host to the client. 
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Figure 3 Variation 3 
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September 27, 2001 

Mr. Roland Hochmuth 
Hewlett-Packard Company 
3404 E. Harmony Road 
Fort Collins, CO 80527 

Re: U.S. Patent Application 

Entitled: SYSTEM AND METHOD FOR 
DISPLAYING AN IMAGE ON A DISPLAY DEVICE 
YourRef. No.: 10017760-1 
Our File No.: 5804.57-1 

U.S. Patent Application 

Entitled: SYSTEM AND METHOD FOR COMMUNICATING 
GRAPHICS IMAGE DATA OVER A COMMUNICATION NETWORK 
YourRef. No.: 10017761-1 
Our File No.: 5804.58 

Dear Roland: 

Enclosed are draft copies of the above-identified applications for patent along with a copy of the 
drawings. Please coordinate with all other inventors of each patent application for their careful review to 
see that the documents accurately and adequately describe the invention. After the review, please fax 
or e-mail any changes or corrections to me. 

Please note that the inventor has a duty to disclose material prior art to the PTO. Such prior art 
includes relevant patents and printed publications, information concerning public use of methods or 
apparatus related to the invention/and information on public use or sales of the invention (or related 
methods or apparatus) made more than a year ago. Failure to disclose such prior art may invalidate 
any patent issuing on the application. 

Should you have any questions or comments, please do not hesitate to contact me. 

Sincerely, 

Anand Gupta 



AG:vc 
Enclosures 



cc: L.Joy Griebenow (via facsimile w/attachments) 

DALLAS 536456 1 5804.57 



